Hey LiveKit team! Running into an issue with LiveKit Agents v1.2.8 when using a vLLM inference server (OpenAI-compatible API). Getting consistent 400 errors: 'Conversation roles must alternate user/assistant/user/assistant/...' Setup: Using openai.LLM() with custom base_url pointing to vLLM server Error happens after a few conversation exchanges vLLM backend rejects requests due to malformed conversation history Error pattern: APIStatusError: Error code: 400 - conversation roles must alternate→ Retries 3x then fails with APIConnectionError Seems like LiveKit might be sending conversation history that doesn't strictly alternate user/assistant roles when talking to vLLM. Works fine initially but breaks mid-conversation. Any known issues with OpenAI-compatible endpoints? Or is there a config param I'm missing for conversation history formatting? Can share full logs if needed! 🙏

Thanks for reaching out, Subhodip. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

(livekit-voice-agent) subh@Jarvis:~/Documents/startups/livekit-voice-agent$ uv run agent.py console 2025-09-12 130121,745 - DEBUG asyncio - Using selector: EpollSelector ================================================== Livekit Agents - Console ================================================== Press [Ctrl+B] to toggle between Text/Audio mode, [Q] to quit. 2025-09-12 130121,746 - INFO livekit.agents - starting worker {"version": "1.2.8", "rtc-version": "1.0.12"} 2025-09-12 130121,746 - INFO livekit.agents - preloading plugins {"packages": ["livekit.plugins.openai", "livekit.plugins.deepgram", "livekit.plugins.silero", "livekit.plugins.turn_detector", "av"]} 2025-09-12 130121,747 - INFO livekit.agents - starting inference executor 2025-09-12 130122,387 - INFO livekit.agents - initializing process {"pid": 13490, "inference": true} 2025-09-12 130122,529 - DEBUG livekit.agents - initializing inference runner {"runner": "lk_end_of_utterance_multilingual", "pid": 13490, "inference": true} None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used. 2025-09-12 130124,545 - INFO livekit.agents - process initialized {"pid": 13490, "inference": true, "elapsed_time": 2.16} 2025-09-12 130124,544 - DEBUG livekit.agents - inference runner initialized {"runner": "lk_end_of_utterance_multilingual", "elapsed_time": 2.014582973999495, "pid": 13490, "inference": true} 2025-09-12 130124,544 - DEBUG asyncio - Using selector: EpollSelector {"pid": 13490, "inference": true} 2025-09-12 130124,556 - INFO livekit.agents - initializing job runner {"tid": 13511} 2025-09-12 130124,556 - DEBUG asyncio - Using selector: EpollSelector 2025-09-12 130124,556 - INFO livekit.agents - job runner initialized {"tid": 13511, "elapsed_time": 0.0} 2025-09-12 130125,809 - DEBUG livekit.agents - http_session(): creating a new httpclient ctx 2025-09-12 130125,812 - DEBUG livekit.agents - using audio io:

ChatCLI

->

AgentSession

->

TranscriptSynchronizer

->

ChatCLI

2025-09-12 130125,812 - DEBUG livekit.agents - using transcript io:

AgentSession

->

TranscriptSynchronizer

->

ChatCLI

2025-09-12 130128,528 - DEBUG livekit.agents - received user transcript {"user_transcript": "Hey. How are you?", "language": "en"} 2025-09-12 130128,591 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.2790285050868988, "input": "|im_start|user\nhey how are you", "duration": 0.039} 2025-09-12 130143,890 - DEBUG livekit.agents - received user transcript {"user_transcript": "So what do you think about", "language": "en"} 2025-09-12 130144,730 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.0004022472712676972, "input": "|im_start|user\nhey how are you|im_end|\n|im_start|assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you|im_end|\n|im_start|user\nso what do you think about", "duration": 0.095} 2025-09-12 130144,923 - DEBUG livekit.agents - received user transcript {"user_transcript": "Indian cricket?", "language": "en"} 2025-09-12 130145,024 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.1645590364933014, "input": "|im_start|user\nhey how are you|im_end|\n|im_start|assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you|im_end|\n|im_start|user\nso what do you think about indian cricket", "duration": 0.1} 2025-09-12 130151,699 - DEBUG livekit.agents - received user transcript {"user_transcript": "So what do you think", "language": "en"} 2025-09-12 130151,833 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.045831482857465744, "input": "|im_start|user\nhey how are you|im_end|\n|im_start|assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you|im_end|\n|im_start|user\nso what do you think about indian cricket so what do you think", "duration": 0.129} 2025-09-12 130153,621 - DEBUG livekit.agents - received user transcript {"user_transcript": "maybe", "language": "en"} 2025-09-12 130154,262 - DEBUG livekit.agents - received user transcript {"user_transcript": "if", "language": "en"} 2025-09-12 130154,493 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.0003597170580178499, "input": "|im_start|user\nhey how are you|im_end|\n|im_start|assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you|im_end|\n|im_start|user\nso what do you think about indian cricket|im_end|\n|im_start|assistant\noh indian cricket that's a huge topic 😄 i|im_end|\n|im_start|user\nso what do you think maybe if", "duration": 0.126} 2025-09-12 130155,515 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.0003597170580178499, "input": "|im_start|user\nhey how are you|im_end|\n|im_start|assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you|im_end|\n|im_start|user\nso what do you think about indian cricket|im_end|\n|im_start|assistant\noh indian cricket that's a huge topic 😄 i|im_end|\n|im_start|user\nso what do you think maybe if", "duration": 0.132} 2025-09-12 130157,973 - DEBUG livekit.agents - received user transcript {"user_transcript": "we do handle", "language": "en"} 2025-09-12 130158,220 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.0010344149777665734, "input": "|im_start|user\nhey how are you|im_end|\n|im_start|assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you|im_end|\n|im_start|user\nso what do you think about indian cricket|im_end|\n|im_start|assistant\noh indian cricket that's a huge topic 😄 i|im_end|\n|im_start|user\nso what do you think maybe if we do handle", "duration": 0.128} 2025-09-12 130200,902 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.0010344149777665734, "input": "|im_start|user\nhey how are you|im_end|\n|im_start|assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you|im_end|\n|im_start|user\nso what do you think about indian cricket|im_end|\n|im_start|assistant\noh indian cricket that's a huge topic 😄 i|im_end|\n|im_start|user\nso what do you think maybe if we do handle", "duration": 0.141} 2025-09-12 130200,920 - DEBUG livekit.agents - received user transcript {"user_transcript": "let's say,", "language": "en"} 2025-09-12 130201,025 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.013820499181747437, "input": "|im_start|user\nhey how are you|im_end|\n|im_start|assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you|im_end|\n|im_start|user\nso what do you think about indian cricket|im_end|\n|im_start|assistant\noh indian cricket that's a huge topic 😄 i|im_end|\n|im_start|user\nso what do you think maybe if we do handle let's say", "duration": 0.102} 2025-09-12 130201,997 - WARNING livekit.agents - failed to generate LLM completion, retrying in 0.1s {"llm": "livekit.plugins.openai.llm.LLM", "attempt": 1} Traceback (most recent call last): File "/home/subh/Documents/startups/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/llm/llm.py", line 177, in _main_task return await self._run() ^^^^^^^^^^^^^^^^^ File "/home/subh/Documents/startups/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/plugins/openai/llm.py", line 822, in _run raise APIStatusError( ...5 lines... ) from None livekit.agents._exceptions.APIStatusError: Error code: 400 - {'object': 'error', 'message': 'Conversation roles must alternate user/assistant/user/assistant/... Conversation roles must alternate user/assistant/user/assistant/...', 'type': 'BadRequestError', 'param': None, 'code': 400} (status_code=400, request_id=None, body={'object': 'error', 'message': 'Conversation roles must alternate user/assistant/user/assistant/... Conversation roles must alternate user/assistant/user/assistant/...', 'type': 'BadRequestError', 'param': None, 'code': 400}, retryable=True) 2025-09-12 130202,552 - WARNING livekit.agents - failed to generate LLM completion, retrying in 2.0s {"llm": "livekit.plugins.openai.llm.LLM", "attempt": 2} Traceback (most recent call last): File "/home/subh/Documents/startups/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/llm/llm.py", line 177, in _main_task return await self._run() ^^^^^^^^^^^^^^^^^ File "/home/subh/Documents/startups/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/plugins/openai/llm.py", line 822, in _run raise APIStatusError( ...5 lines... ) from None livekit.agents._exceptions.APIStatusError: Error code: 400 - {'object': 'error', 'message': 'Conversation roles must alternate user/assistant/user/assistant/... Conversation roles must alternate user/assistant/user/assistant/...', 'type': 'BadRequestError', 'param': None, 'code': 400} (status_code=400, request_id=None, body={'object': 'error', 'message': 'Conversation roles must alternate user/assistant/user/assistant/... Conversation roles must alternate user/assistant/user/assistant/...', 'type': 'BadRequestError', 'param': None, 'code': 400}, retryable=True) 2025-09-12 130203,606 - DEBUG livekit.agents - received user transcript {"user_transcript": "something else.", "language": "en"} this was the error

I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!

(livekit-voice-agent) subh@Jarvis:~/Documents/startups/livekit-voice-agent$ uv run agent.py console 2025-09-12 130121,745 - DEBUG asyncio - Using selector: EpollSelector ================================================== Livekit Agents - Console ================================================== Press [Ctrl+B] to toggle between Text/Audio mode, [Q] to quit. 2025-09-12 130121,746 - INFO livekit.agents - starting worker {"version": "1.2.8", "rtc-version": "1.0.12"} 2025-09-12 130121,746 - INFO livekit.agents - preloading plugins {"packages": ["livekit.plugins.openai", "livekit.plugins.deepgram", "livekit.plugins.silero", "livekit.plugins.turn_detector", "av"]} 2025-09-12 130121,747 - INFO livekit.agents - starting inference executor 2025-09-12 130122,387 - INFO livekit.agents - initializing process {"pid": 13490, "inference": true} 2025-09-12 130122,529 - DEBUG livekit.agents - initializing inference runner {"runner": "lk_end_of_utterance_multilingual", "pid": 13490, "inference": true} None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used. 2025-09-12 130124,545 - INFO livekit.agents - process initialized {"pid": 13490, "inference": true, "elapsed_time": 2.16} 2025-09-12 130124,544 - DEBUG livekit.agents - inference runner initialized {"runner": "lk_end_of_utterance_multilingual", "elapsed_time": 2.014582973999495, "pid": 13490, "inference": true} 2025-09-12 130124,544 - DEBUG asyncio - Using selector: EpollSelector {"pid": 13490, "inference": true} 2025-09-12 130124,556 - INFO livekit.agents - initializing job runner {"tid": 13511} 2025-09-12 130124,556 - DEBUG asyncio - Using selector: EpollSelector 2025-09-12 130124,556 - INFO livekit.agents - job runner initialized {"tid": 13511, "elapsed_time": 0.0} 2025-09-12 130125,809 - DEBUG livekit.agents - http_session(): creating a new httpclient ctx 2025-09-12 130125,812 - DEBUG livekit.agents - using audio io:

ChatCLI

->

AgentSession

->

TranscriptSynchronizer

->

ChatCLI

2025-09-12 130125,812 - DEBUG livekit.agents - using transcript io:

AgentSession

->

TranscriptSynchronizer

->

ChatCLI

2025-09-12 130128,528 - DEBUG livekit.agents - received user transcript {"user_transcript": "Hey. How are you?", "language": "en"} 2025-09-12 130128,591 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.2790285050868988, "input": "<|im_start|>user\nhey how are you", "duration": 0.039} 2025-09-12 130143,890 - DEBUG livekit.agents - received user transcript {"user_transcript": "So what do you think about", "language": "en"} 2025-09-12 130144,730 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.0004022472712676972, "input": "<|im_start|>user\nhey how are you<|im_end|>\n<|im_start|>assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you<|im_end|>\n<|im_start|>user\nso what do you think about", "duration": 0.095} 2025-09-12 130144,923 - DEBUG livekit.agents - received user transcript {"user_transcript": "Indian cricket?", "language": "en"} 2025-09-12 130145,024 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.1645590364933014, "input": "<|im_start|>user\nhey how are you<|im_end|>\n<|im_start|>assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you<|im_end|>\n<|im_start|>user\nso what do you think about indian cricket", "duration": 0.1} 2025-09-12 130151,699 - DEBUG livekit.agents - received user transcript {"user_transcript": "So what do you think", "language": "en"} 2025-09-12 130151,833 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.045831482857465744, "input": "<|im_start|>user\nhey how are you<|im_end|>\n<|im_start|>assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you<|im_end|>\n<|im_start|>user\nso what do you think about indian cricket so what do you think", "duration": 0.129} 2025-09-12 130153,621 - DEBUG livekit.agents - received user transcript {"user_transcript": "maybe", "language": "en"} 2025-09-12 130154,262 - DEBUG livekit.agents - received user transcript {"user_transcript": "if", "language": "en"} 2025-09-12 130154,493 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.0003597170580178499, "input": "<|im_start|>user\nhey how are you<|im_end|>\n<|im_start|>assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you<|im_end|>\n<|im_start|>user\nso what do you think about indian cricket<|im_end|>\n<|im_start|>assistant\noh indian cricket that's a huge topic 😄 i<|im_end|>\n<|im_start|>user\nso what do you think maybe if", "duration": 0.126} 2025-09-12 130155,515 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.0003597170580178499, "input": "<|im_start|>user\nhey how are you<|im_end|>\n<|im_start|>assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you<|im_end|>\n<|im_start|>user\nso what do you think about indian cricket<|im_end|>\n<|im_start|>assistant\noh indian cricket that's a huge topic 😄 i<|im_end|>\n<|im_start|>user\nso what do you think maybe if", "duration": 0.132} 2025-09-12 130157,973 - DEBUG livekit.agents - received user transcript {"user_transcript": "we do handle", "language": "en"} 2025-09-12 130158,220 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.0010344149777665734, "input": "<|im_start|>user\nhey how are you<|im_end|>\n<|im_start|>assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you<|im_end|>\n<|im_start|>user\nso what do you think about indian cricket<|im_end|>\n<|im_start|>assistant\noh indian cricket that's a huge topic 😄 i<|im_end|>\n<|im_start|>user\nso what do you think maybe if we do handle", "duration": 0.128} 2025-09-12 130200,902 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.0010344149777665734, "input": "<|im_start|>user\nhey how are you<|im_end|>\n<|im_start|>assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you<|im_end|>\n<|im_start|>user\nso what do you think about indian cricket<|im_end|>\n<|im_start|>assistant\noh indian cricket that's a huge topic 😄 i<|im_end|>\n<|im_start|>user\nso what do you think maybe if we do handle", "duration": 0.141} 2025-09-12 130200,920 - DEBUG livekit.agents - received user transcript {"user_transcript": "let's say,", "language": "en"} 2025-09-12 130201,025 - DEBUG livekit.plugins.turn_detector - eou prediction {"eou_probability": 0.013820499181747437, "input": "<|im_start|>user\nhey how are you<|im_end|>\n<|im_start|>assistant\nhey there as an ai i don't experience things like being in the way people do but i'm functioning optimally and ready to help so you could say i'm doing well 😄 how are you<|im_end|>\n<|im_start|>user\nso what do you think about indian cricket<|im_end|>\n<|im_start|>assistant\noh indian cricket that's a huge topic 😄 i<|im_end|>\n<|im_start|>user\nso what do you think maybe if we do handle let's say", "duration": 0.102} 2025-09-12 130201,997 - WARNING livekit.agents - failed to generate LLM completion, retrying in 0.1s {"llm": "livekit.plugins.openai.llm.LLM", "attempt": 1} Traceback (most recent call last): File "/home/subh/Documents/startups/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/llm/llm.py", line 177, in _main_task return await self._run() ^^^^^^^^^^^^^^^^^ File "/home/subh/Documents/startups/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/plugins/openai/llm.py", line 822, in _run raise APIStatusError( ...<5 lines>... ) from None livekit.agents._exceptions.APIStatusError: Error code: 400 - {'object': 'error', 'message': 'Conversation roles must alternate user/assistant/user/assistant/... Conversation roles must alternate user/assistant/user/assistant/...', 'type': 'BadRequestError', 'param': None, 'code': 400} (status_code=400, request_id=None, body={'object': 'error', 'message': 'Conversation roles must alternate user/assistant/user/assistant/... Conversation roles must alternate user/assistant/user/assistant/...', 'type': 'BadRequestError', 'param': None, 'code': 400}, retryable=True) 2025-09-12 130202,552 - WARNING livekit.agents - failed to generate LLM completion, retrying in 2.0s {"llm": "livekit.plugins.openai.llm.LLM", "attempt": 2} Traceback (most recent call last): File "/home/subh/Documents/startups/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/agents/llm/llm.py", line 177, in _main_task return await self._run() ^^^^^^^^^^^^^^^^^ File "/home/subh/Documents/startups/livekit-voice-agent/.venv/lib/python3.13/site-packages/livekit/plugins/openai/llm.py", line 822, in _run raise APIStatusError( ...<5 lines>... ) from None livekit.agents._exceptions.APIStatusError: Error code: 400 - {'object': 'error', 'message': 'Conversation roles must alternate user/assistant/user/assistant/... Conversation roles must alternate user/assistant/user/assistant/...', 'type': 'BadRequestError', 'param': None, 'code': 400} (status_code=400, request_id=None, body={'object': 'error', 'message': 'Conversation roles must alternate user/assistant/user/assistant/... Conversation roles must alternate user/assistant/user/assistant/...', 'type': 'BadRequestError', 'param': None, 'code': 400}, retryable=True) 2025-09-12 130203,606 - DEBUG livekit.agents - received user transcript {"user_transcript": "something else.", "language": "en"} this was the error (edited)